Method and apparatus for processing, displaying and communicating images

ABSTRACT

The application discloses an apparatus and method of processing images, hand-drawn or written on a suitable Writing Surface, viewed by an Image Sensor such as a video camera, and captured by an image sensing circuit (i.e., frame grabber or similar) used for acquisition of image frames by a computer. More particularly, this invention discriminates among changes detected in these Viewed Images in order to identify and disregard non-informational, transient and/or redundant content. Removal of such content, a writer&#39;s arm for example, from the captured image facilitates isolating meaningful changes, specifically intentional new Writings and Erasures appearing on the Writing Surface. Preserving only meaningful changes on the surface promotes optimized: “video image” storage and compressed transmission of a subset of the visual data, when used in conjunction with digital computers at remote locations optionally equipped with projected image and/or conventional computer display systems. The invention provides for displaying a composite of the remote sites&#39; changes, transmitted over conventional communications channels, omitting Local Markings where the display device is a projector. The preferred embodiment of the present invention is applied to remote presentation aids, in particular “whiteboards” such as those typically used in educational lectures and commercial sales and training settings, and in particular to remote whiteboards. As will become clear, however, the application of the present invention is not restricted to whiteboards exclusively, but can work with any Writing Surface suitable for image acquisition (e.g., flipcharts or paper on a desk). Receive-only sites may participate in the remote presentation without contributing Local Markings to the composite. The present invention is compatible with conventional software products which will carry audio and other data streams on the same channel as the compressed image signals.

CROSS-REFERENCE TO RELATED APPLICATION

This is an application pursuant to a provisional application under thetitle “Remote Virtual Whiteboard,” filed Oct. 3, 1997 and assigned Ser.No. 60/060,942.

FIELD OF THE INVENTION

The present invention relates generally to the field of processingimages, and more particularly, to the field of remote conferencingapplications.

INCORPORATION BY REFERENCE

This application incorporates by reference the “Interactive ProjectedVideo Image Display System” disclosed under U.S. Pat. No. 5,528,263(Platzker et al.) as if set forth at length herein.

Definitions

Background Image A computer-generated template, often from anotheractive application (such as Microsoft® PowerPoint®¹), incorporated intothe Computer Display Image.

¹ Microsoft and PowerPoint are registered trademarks of MicrosoftCorporation.

A Committed Image A composite of the Background Image and all WritingImages to be saved into some suitable medium, such as storage component,transmission channel, or to a compatible software application.

Computer Display Image The display presented by a computer for humanviewing on a device such as a monitor or Projection Surface employingtechnologies such as VGA, SVGA, XGA and others.

Image Sensor An optical device such as a video camera, digital camera orother imaging technology capable of viewing a Writing Surface with LocalMarkings.

Local Markings Markings (“Local Writings” and/or “Local Erasures”) madeon the Writing Surface of the site of reference.

Local Updates Stream of data packets containing compressedrepresentations of changes made to Local Markings.

Markings Writings and annotations (collectively, “Writings”) and/ordeletions (“Erasures”) made by a human presenter on a Writing Surface.

Projection Surface A surface upon which the Computer Display Imagecreated by a computer-controlled projection may appear; in the presentinvention, this will substantially overlap the Writing Surface.

Projections Visual information that appears in the Viewed Image as aresult of projecting the Computer Display Image onto the Writing Surface(distinguished from Markings and Viewed Image Interference); in thecontext of the present invention Projections are relevant insomuch asthey appear similar to physical Markings or otherwise complicate thetask of detecting and processing Markings.

Remote Markings Markings (“Remote Writings” and/or “Remote Erasures”)made on the Writing Surfaces of non-local sites.

Remote Updates Stream of data packets containing compressedrepresentations of changes made to Remote Markings.

Stored Viewed Image The most recent Viewed Image modified to excludeViewed Image Interference and retained for purposes of detecting changesto Local Markings in a comparison with a Viewed Image.

Stored Writing Image The most recent Writing Image retained for purposesof encoding Local Updates in a comparison with an updated Writing Image.

Viewed Image The image acquired (or “seen”) by the Image Sensor and madeavailable as digital information to computational resources(software/hardware).

Viewed Image Interference Physical objects that are interposed betweenthe Image Sensor and the Writing Surface and therefore appear in theViewed Image, for example, a writer's arm or body (distinguished fromMarkings and Projections).

Warping A transformation performed on an image based on a mappingbetween two geometrical coordinate spaces; in the present inventionViewed Images (or portions thereof) are transformed in this manner to apredefined display coordinate space and projected images (or portionsthereof) are transformed to the coordinate space of the Viewed Images(using both Warping and optional “scaling” to overcome differences inpixel-resolution); the geometric mapping is obtained through a processof calibration.

Writing Images Internal representations of the Local Markings or RemoteMarkings at any point in time, one Writing Image per participatingtransmitting site.

Writing Surface Any surface suitable for human writing, such as awhiteboard, paper flip-chart or sheets of paper; in the presentinvention, this will also be the Projection Surface, if employed.

BACKGROUND OF THE INVENTION

Business, academic and other professional meetings are held to impartinformation to, and solicit ideas from, the attendees. The convener of ameeting usually seeks participation from the attendees, and it is aneffective use of expensive meeting time to capture and record the keythoughts and ideas of the presenters and participants for futurereference. It is particularly effective to record and display keypoints, numbers, etc., dynamically as these are forthcoming fromparticipants at a meeting or seminar. An issue is how to record anddisplay this information most effectively with minimal distraction. Thetranscription by hand of information written on a vertical board isknown to every school child.

Such data are ephemeral, however, and must eventually be erased.Traditionally, each participant takes his or her own notes, each copyingessentially the same material. Paradoxically, valuable information maybe missed by one or several or many of participants because of thediversion of their attention to note-taking, and there may be errors inthe transcription.

One approach to solving this problem is disclosed in the U.S. Pat. No.5,528,263 previously incorporated hereinabove by reference, the“Interactive Projected Video Image Display System.” In this patentbeginning at column 6, line 60 a function is described to enable acomputer to capture updated images. This system has been commercializedas Digital Flipchart™ (“DFC”). The system disclosed in the Platzker etal. patent and the one commercialized by Tegrity can be used at only onelocation and cannot capture an updated image without blanking the screenand cannot ignore transient objects in the field of view.

Video technology could solve the problem and limitations of the priorart, but would increase the cost and complexity of linking remote sites.Linking remote sites would allow remote participants to view writteninformation from other sites in real-time. Using video technology tothis end, the presenter must be careful not to block the view unduly,adding an unnatural constraint to a presentation. On the remote end,video images of a person's arm, blouse or tie are unnatural,unsatisfactory, uninformative, distracting at best and, at worst, anannoyance that detracts from the value of the presentation to remoteviewers. Moreover, such useless video data further burdens thetransmission channel. Combining video images from dispersed locationsadds even more complexity.

Transmitting even compressed video uses substantial bandwidth.Transmitting compressed video of images containing motion uses even morebandwidth.

SUMMARY OF THE INVENTION

The present invention is an apparatus and method for providing acomposite image on a standard whiteboard, flipchart or other WritingSurface's location at a first site, consisting of Local Markings and aprojection of only the meaningful changes (i.e., Writings and Erasures)made to at least a second Writing Surface located at a second site. Theresult is an appearance on the local Projection Surface of a combinationof physical Local Markings together with a projection of the remotesites' respective Markings. Thus, the same composite image appears onthe Projection Surface at every site, although local viewers see theactual physical Markings made locally, as opposed to the projectedRemote Markings. To human perception, everyone at every connected siteis seeing the “same” image, differing only in size and to the degreeimposed by the technical capabilities of the local projector. Thecomposite image also incorporates the Background Image if one isgenerated within the computer.

In general, each site may utilize an Image Sensor connected to astandard PC-type processor. The process, when employed in such a site,continuously “looks” (i.e., grabs Viewed Images via the Image Sensor) atthe present composition of the Writing Surface. It also continuouslymonitors the present composition of the Computer Display Image, if it isprojected on the Writing Surface. These image are analyzed to detect theexistence and precise locations of three kinds of information:

1. Viewed Image Interference—these areas are detected, ignored andtherefore not encoded or transmitted and no bandwidth is consumed bythem.

2. Projected “writings”— information that represents projected “objects”sensed by the Image Sensor (e.g., Writings from other stations which areprojected by a projector). These Projections are similarly detected,ignored and withheld from transmission.

3. Local changes—changes to Local Markings which appeared after theprevious analysis. These local changes are processed. Remote sites (ifany) may utilize a different display resolution (the number of pixels inthe VGA). Local changes undergo a process of geometric adaptation(Warping), which transforms them to a common display resolution. Thelocal changes are then encoded in a compressed format as Local Updatesand sent to other, remote sites for display (with optional buffering ofthe transmitted data), and/or appended to a stored data stream. Theresult of the process is that each site displays the Writings of allother parties. This display may be projected on the same physicalProjection Surface upon which Local Markings are written. Consequently,all of the displays at each interconnected site contain all of theinformation recorded elsewhere. Local Writings (i.e., locally recordednotations) are not displayed (or projected). Users have a “commitcapture” option whereby the resulting composite Committed Image of theWritings from all sites may be saved. There may also be “receiver” onlystations, i.e., sites which do not transmit Local Updates, if any, andneed not be equipped with an Image Sensor or a Writing Surface. Thesesites merge and display Markings received from other physical sites.Optionally, such sites can be made to operate using standard, widelyavailable Internet browsing software without the need for additional,special-purpose hardware or software components.

In an optional embodiment, a remote site can be configured asdisplay-only. The composite image for display or projection at such adisplay-only (or receive-only) site would consist of the assembled,ongoing changes to Remote Markings and Background Image without anyphysical Local Markings comprising part of the image.

The remote conferencing capabilities of the invention are facilitated bythe innovation of a method of using a computer to isolate changes toMarkings, filtering out transient data, and compressing the result fortransmission and/or storage. The computer processes the Viewed Imagesignals, or “frames,” representing the images appearing in the viewingfield of the local Image Sensor indicative of Markings made on the localWriting Surface. The meaningful portion of this continuous sequence offrames are those not indicative of the presence of Viewed ImageInterference, should any appear transiently in the local viewing field,or of Projections. The method detects changes between successive frames.These frames are filtered to reject those signals below a predeterminedspatial frequency representing obstructions, and the successive,filtered frames are further examined to detect changes indicative of theWritings or Erasures made on the local Writing Surface as distinguishedfrom transient changes characteristic of Viewed Image Interference.These changes are further examined to eliminate those that are caused bychanges to the Projections, if the Computer Display Image is projected.The result is a compressed representation of changes to Markings,suitable for decompression and display at the receiving end. As such,the method may be termed a “writing codec.”

Representations of the successive changes may be stored for laterplayback, locally or remotely. In one embodiment, an image may betransmitted from time to time even without changes to permit a latejoining site or one which is restarting to develop a composite image.

DESCRIPTION OF THE DRAWINGS

The present invention is more readily understandable by reference to thefollowing detailed description read in pari materia with theaccompanying drawings in which:

FIG. 1 is an example of variously configured, interconnectedtransmit-and-receive and receive-only sites utilizing the imageprocessing capabilities of the present invention.

FIG. 2A is a model of a typical configuration depictingtransmit-and-receive site that operates in accordance with the presentinvention.

FIG. 2B is a model of a typical configuration depicting the requiredcomponents for a receive-only site that operates in accordance with thepresent invention.

FIG. 3 is a high-level flowchart of the preferred embodiment of themethod of the image processing of the present invention.

FIG. 4A is a representation of a Writing Surface appearance throughoutthe progression of a hypothetical session as perceived by a humanobserver in multiple, interconnected transmit-and-receive sites A, B andC, each of which are equipped with Image Sensors.

FIG. 4B is a representation of the progression of the composite ComputerDisplay Image of the same hypothetical session of FIG. 4A as displayedat receive-only sites D (with projector-type display device) and E (withcomputer monitor-type display device).

FIG. 5 is a detailed depiction of the process by which one of the LocalUpdates represented in FIG. 4A at transmit-and-receive site B is readiedfor transmission and/or storage.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT OF THE INVENTION

Referring now to FIG. 1, there is a drawing depicting a plurality ofimage processing sites (A through E), interconnected by a communicationinfrastructure [11]. Five sites are shown. Sites A, B and C each havetwo independent operating modes: Transmit and Receive. These sites aresources of transmitted images because each is equipped with an ImageSensor [22] to view the Markings on their respective Writing/ProjectionSurfaces [21]. Each such site A, B and C can operate in either Receivemode, Transmit mode or both simultaneously, and may be referred to forconvenience as “transmit-and-receive” sites. The other two sites, D andE, are not equipped with image-sensing devices and thus operate inReceive mode only (“receive-only” sites). In the event that Markings aremade upon the Projection Surface [21D] at site D, these Markings willnot be input, processed or transmitted to any other site. All sites maybe linked by virtually any type of data communication infrastructure[11], for example, via modem, local area network (“LAN”), Internet orother types of communications channel.

FIGS. 2A and 2B depict in more detail the typical site configurationsfor transmit-and-receive and receive-only implementations of the presentinvention. FIG. 2A shows the major components required for usingTransmit and Receive mode. The site in FIG. 2A has an Image Sensor [22]to capture images of a local Writing Surface [21] continuously into acomputer [23]. A projector [24] projects the computer generated ComputerDisplay Image onto the Writing Surface [21]. A person can also write anderase Markings on the Writing Surface [21] interspersed with theprojected image.

FIG. 2B depicts a configuration for receive-only sites D and E, withoutthe optional Image Sensor [22] and corresponding video input circuit incomputer [23] required at transmit-and-receive sites A, B and C. Areceive-only site may be configured the same as a transmit-and-receivesite, but in that event these devices will not be utilized by thepresent invention. At a receive-only site, the display output may bedirected to a projector-type device or a typical computer monitor (suchas [21D] and [12] depicted in FIG. 1 at sites D and E, respectively).

In each instance of a transmit-and-receive (FIG. 2A) and receive-onlysite (FIG. 2B), the computer [23] is connected to a communicationschannel [25]. In another embodiment of the present invention (notdepicted), the session may be played back from remote storage device viathe communications channel [25]. In addition, another embodimentprovides for the playback of a session from a local data storage device[26], in which case the communications link [25] would not be required.

FIG. 3 is a flow diagram of the major components of the method wherebythe various raw and processed images deriving from local and remotesources, are manipulated in accordance with the teachings of thepreferred embodiment of this invention. The implementation of thepreferred embodiment is accomplished in software, resident and operatingin the computer device [23] depicted in FIGS. 1, 2A and 2B. Areceive-only site (FIG. 2B and sites D and E in FIG. 1) requires neitherthe functionality of the “capture engine” [31], as the local inputsshown as [301], [302] and [303] would not be present, nor of the “Encodelocal update” subprocess of the “capture codec”[34], as there would beno local data incoming from [31] for it to process. Accordingly, in oneembodiment of the present invention, these functions may be omitted inreceive-only implementations.

FIGS. 4A and 4B are illustrative examples of a sequence of illustrationsthat represent human perception of the Writing Surfaces of sites A, Band C (shown in FIG. 1 as [21A] through [21C]), as well as theProjection Surface [21D] of site D and monitor [12] of site E, as aresult of using the computers at each site ([23] of FIGS. 1, 2A and 2B)to execute the process of the present invention (diagramed in FIG. 3).The diagram of FIG. 4A shows successive “snapshots” in time of the threeremote transmit-and-receive sites A, B and C of FIG. 1. Each framedepicts a point in time where there has been detected a meaningfulchange to the Markings on at least one of the transmit-and-receivesites' Writing Surfaces. Each frame is numbered from [41] through [46],inclusive, in ascending time sequence, with a suffix representing thesite. Frames labeled with the same integer but a differing suffix (e.g.,[41A] and [41C]) depict the simultaneous appearance of the Writingand/or display Surfaces at the respective sites.

For purposes of illustration, the rectangular frames represent theperiphery of the Writing Surfaces at transmit-and-receive sites A, B andC ([21] in FIGS. 1 and 2A) upon which the Image Sensors should betrained. In the case of the receive-only site D, the rectanglerepresents the periphery of the Projection Surface ([21D] in FIG. 1); atsite E, the shadowed frame represents the edge (or, outermost pixels) ofthe monitor ([12] in FIG. 1).

Further with respect to FIG. 4A, certain figures are superimposed uponthese rectangles: a stylized person moving in front of the illustrationsat time sequences [41A], [43A] and [46A], and a hand with the indexfinger extended at time sequences [42B] and [45B]. These superimposedfigures represent undesirable, transient obstructions (Viewed ImageInterference) in the fields of view of the Image Sensors [22A] and [22B]at sites A and B, respectively. Images of obstructions processed by thecomputers [23] at sites A and B are removed by the execution of theprocess diagramed in FIG. 3 and, therefore, are not transmitted over thecommunications channels [25] and do not appear on the displays/surfacesillustrated at any other site.

FIG. 4B depicts the resulting overall composite image which would bedisplayed at receive-only sites. Each sequence would appear on theProjection Surface (shown as [21D] in FIG. 1) at site D and upon themonitor display ([12] in FIG. 1) at site E. The composite imageappearing at any receive-only site would be the same, except for anygeometrical adjustment such as for resolution or scale. Thesegeometrical adjustments are performed locally. Thus the displaysappearing on [21D] and [12] at receive-only sites D and E, respectively,are themselves composites replicating all of the Local Markings of alltransmit-and-receive sites A, B and C. It is possible that receive-onlysite D may use a whiteboard (or other Writing Surface) as its ProjectionSurface [21D]. In the event that any Local Markings happen to be made atsite D upon that Projection Surface [21D], these Markings will not bedetected or processed in Receive-mode, and will not be incorporated inthe overall composite image. In other words, if Local Markings are madeat receive-only site D, these Markings do not appear at any other sitebecause site D does not operate in transmit mode.

FIG. 5 further illustrates the result of the process of FIG. 3,performed by the computer [23]. In particular, FIG. 5 details theprocess whereby transmit-and-receive site B produces the Local Updatethat corresponds to the change shown in FIG. 4A from time sequence [44](illustrated at transmit-and-receive site C as [44C]) to time sequence[45] (depicted in FIG. 4A at transmit-and-receive sites B and C as [45B]and [45C], respectively).

In accordance with the present invention, there is no inherentrestriction on the number of sites which may participate in a real-timesession. External constraints such as limitations of computer memory orprocessing or channel capacity may make an arbitrary limitation on thenumber of participating sites desirable. There must be at least one siteoperating in Transmit mode, except in the instance of a session beingplayed back from a storage device [26].

In another embodiment of the present invention, not herein depicted,there need not be any sites operating in Receive mode, where the sessionis being recorded to storage [26] for the purpose of later playback. Insuch an embodiment, the communications channel [25] is not required. Ina variant of this embodiment, the ongoing changes to the Writing Imageneed not be recorded to storage [26] continuously as Local Markings aremade, but may be committed to storage [26] and/or be displayed [12] orprojected [24] intermittently upon demand, as with capture function ofthe DFC product previously described. Advantages of such an embodimentover the DFC implementation include avoidance of the blanking of theComputer Display Image, more immediate availability of the CommittedImage, and additional accuracy inherent in monitoring the progression ofLocal Markings over time.

In still another embodiment of the present invention, also not depicted,any of the transmit-and-receive sites A, B and C of FIG. 1 could beoperated in Transmit mode alone. Such a site would be deemed to be atransmit-only site, without the capability of receiving and displayingRemote Updates. A projector [24] would be optional, depending uponwhether the projection of a Background Image is desired. In yet anotherembodiment, a transmit-and-receive or transmit-only site could operatewithout a projector [24].

In operation, certain preliminary steps must be performed in order toeffect the teachings of the invention. This disclosure assumes thatprojection devices [24] at any site have been properly focused upon aProjection Surface [21]. Where the projection device is also the WritingSurface at a transmit-and-receive site ([21A], [21B] and [21C]), it isrequired that the field of view of the site's Image Sensor [22] betrained upon, and approximately aligned with, the periphery of itsProjection Surface [21]. It is advisable to have the Image Sensor [22]and projector [24] in the closest proximity practicable.

To accomplish these, the Image Sensor should be optimally focused ateach site. The focusing of projection and Image Sensors is well known,and is usually performed manually to the user's satisfaction, orperformed automatically if that capability is available in the specificmodel of the local equipment being used.

In addition, the image on the Projection Surface [21] as seen by theImage Sensor [22] must be closely aligned with the image being displayedby the projector [24] upon the same Projection Surface [21]. The presentinvention employs a proprietary technique used by Tegrity's DFC product,wherein the user performs a manual alignment of the Image Sensor [22]and projector [24]. This alignment step involves using the computer [23]to output to the local projector [24] an image of a rectangular testpattern for display upon the Projection Surface [21]. This test patterncontains an inner rectangle within which appears the unedited ViewedImage seen simultaneously by the Image Sensor [22] (scaled down to fitthe size of the rectangle). In other words, the projected test patterncontains a projection of what the Image Sensor [22] “sees.” The userthen physically adjusts the orientation and focus of the Image Sensor[22] until a full replica of the projected test pattern appears withinthe inner rectangle, at which point the alignment is sufficient toperform the process of the present invention. This technique is known inthe prior art.

Once aligned, the system must be calibrated in order to quantifycharacteristics of the Viewed Image seen by the Image Sensor [22].Calibration is necessary because distortions are inevitably introducedby the optics of the Image Sensor [22] and projector [24] (e.g., bylenses) and a parallax effect caused by the distance between the ImageSensor [22] and projector [24] and/or the angle of projection and/or anon-plumbed Writing Surface [21], or some combination of these.

Calibration information [303] may be required as input to allowcomparison between images captured during a session to one or more“reference” images and for adjusting environment-dependent systemparameters. Reference images may be captured before the session startsor at some other time when the system can assume a controlledenvironment (e.g., the Writing Surface is assumed to be clear). Systemparameters may include information about focus, color-balance and othercharacteristics of the input images, as well as information about thecomputer display—such as its pixel resolution. When projection isemployed, the calibration information also includes means ofcomputationally transforming pixel coordinates between the coordinatespace of the projected image (Computer Display Image) and that of theViewed Image. This transformation may combine “scaling” betweendifferent pixel resolutions and Warping, which compensates for thegeometric distortions described above.

The calibration algorithm may be implemented by projecting predeterminedimages that include features with known locations, scale dimensions andcolor or light intensities, capturing Viewed Images and processing themto locate and/or analyze the appearance of the predetermined features.The location of projected targets can be used as inputs to calculate thecomputational parameters of the Warping transformation described above.Techniques for locating projected targets, analyzing them to extractbasic characteristics and computing Warping transformations based ontheir locations in the Viewed Image are well documented inimage-processing literature.

When projection [24] is not employed, reference images may nonethelessbe captured, saved and analyzed to produce the system parameters.Optionally, the user may be instructed to aid in this process, forexample by aiming the Image Sensor [24], clearing the Writing Surface[24] and removing Viewed Image Interference (e.g., stepping aside).Calculating a Warping transformation may be accomplished withoutprojection (although it is not necessarily required in this case) byinstructing the user to, for example, mark corners of the writing areaor to manipulate an image displayed on the computer monitor (e.g., bydrawing a polygon that bounds the writing area as seen in a Viewed Imageshown to the user).

The calibration procedure may contain self-checks that determine whetherthe system is properly aligned and capable of performing its intendedfunctions. The user can be notified when problems are detected and whatthe probable nature of these problems is. For example, a focus-checkdetermines if the Viewed Image has sufficient acuity and requests theuser to adjust the focus if the test fails. As another example, if thereare excessive Markings or Viewed Image Interference in the capturedreference images, the user may be reminded to clear the Writing Surfaceand remove obstructions. These techniques are known in the prior art.

The preferred embodiment of the present invention employs thecalibration sequence of Platzker et al, incorporated by reference,described in column 10, lines 50 through 66, as implemented in the DFCproduct.

With alignment and calibration completed, the innovation of the presentinvention may be performed. The transmit-and-receive sites A, B and Cwhen operating in Transmit mode will produce a “capture update stream”which may be sent to remote sites over a communication link [25] and/orstored locally [26]. This stream consists of Local Updates thatrepresent periodic changes as Markings are being written (and erased) bythe user(s). Receiver mode receives capture update streams from one ormore remote sites or as played back from storage, merges theinformation, and displays it as the local Computer Display Image.Receiver mode requires no special devices other than the computer. Atany given time the user(s) may choose to “commit” the Writing SurfaceWritings so as to freeze and save the Committed Image that representsall participating Writing Surfaces for the current “page” into somesuitable medium, such as storage component [26], transmission channel[25], or to a compatible software application.

Modules Comprising the Steps of the Preferred Embodiment

An implementation of the preferred embodiment includes software with thefollowing components, although variations in the design are possiblewithout departing from the spirit of the invention. The flowchart inFIG. 3 shows these modules and how they are interrelated:

a. Capture Engine [31]—Controls the operation of the software modulesthat track, analyze and report changes to the Writing Surface. Itmanages the timing of the processing and the flow of data betweenmodules.

b. Real-time Detector [32]—Analyzes Viewed Images at a rapid rate todetect events in the visual field. Specifically, the module contains an“interference detector” which determines if and where Viewed ImageInterference appears in the image, a “changes detector” which determinesif and where Markings appear to have changed (written and/or erased) anda “projection cancellation” unit which determines if and where relevantProjections that appear in the Viewed Image have changed so that theymay be discarded. By comparing the results of these units this modulecan determine which changes in the Viewed Image represent actual, LocalMarkings (Writings and Erasures).

c. Capture Processor [33]—When Local Markings have occurred, this moduleperforms image processing of the modified data. Various techniques areemployed to clean-up the image, enhance its appearance for visualizationand to transform it geometrically to a predetermined coordinate space(Warping). The result is an update to the local Writing Image.

d. Capture Codec [34]—This module encodes updates to the local WritingImage in a compressed format as Local Updates for transmission and/orstorage (Transmit mode) and decodes incoming Remote Updates from remotesites or from a playback source (Receive mode). Outgoing Local Updatesmay also undergo splitting into partial Local Updates so as to meetbandwidth limitations that may be imposed externally. This module alsoprovides services of merging and displaying images that combine theWriting Images from the different participating Writing Surfacesincluding scaling the image geometry to match the target computerdisplay resolution as necessary.

Receive mode typically requires only the Capture Codec module [34] whileTransmit mode requires the other modules in addition. An alternativemode of operation allows the Transmitting station to send streams ofLocal Updates that can be viewed by existing Internet browsers withoutrequiring any special-purpose components, such as Capture Codec. Thefollowing describes the operational flow of the process modeled in FIG.3 in greater detail.

Description of Image Processing

This section describes the inputs and outputs of the image process andthe internal operation of each of its modules in the preferredembodiment of the invention as depicted in FIG. 3.

Inputs and Outputs:

The inputs of a transmitter site include up to two continuous streams ofinformation—Viewed Images [301] and updates being made to the ComputerDisplay Images [302]. The former stream [301] contains the Markings thatwill be extracted, processed and transmitted. The optional latter stream[302] is required so that the system may distinguish between changes tophysical Local Markings and changes to Projections if the configurationincludes a projector [24]. If the Computer Display Image is notprojected onto the Writing Surface [21], this stream is not required (asdepicted by the dashed lines in FIG. 3). Otherwise, it can preferably beproduced by trapping events that cause the display to be updated or byperiodically obtaining an internal representation of the ComputerDisplay Image. In either case, modern computer operating systems providethe services, such as callable Microsoft Windows® functions, requiredfor producing this information. In addition, calibration information[303], described above, is also used by a transmitter site.

A transmitter produces as output a stream of Local Updates [304]. TheseLocal Updates are data packets [304] that encode successive changes toLocal Markings; i.e. the Writing and Erasures that takes place on thelocal Writing Surface. Typically, each packet encodes a “delta,”describing the net change to the Writing Surface. When it isadvantageous, other types of packets may be used. For example, when alarge amount of Erasures is detected it may be preferable to produce afull encoding of the Writing Surface or when it is determined that theWriting Surface has been completely cleared a concise “clear” packet maybe used.

Receiver stations accept Remote Update packets produced by transmitters[305] where each packet is labeled with the identification of theoriginating transmitter. There are generally two local outputs from atypical receiver. One output is a composite image that merges therepresentations of all participating Writing Images for the purposes ofstoring a Committed Image [306]. A second output from a receiver is acomposite image that merges all remote Writing Images, and excludes thelocal Writing Image, if present, for the purposes of producing a newComputer Display Image [302]. In some applications, such as whenprojection is used [24] it may be useful to include the Background Image[307] (e.g., containing a spreadsheet, text, diagrams, pictures etc.)when producing either of these composite images. Composite images arecreated by merging any number of input images (Writing Images andoptional Background Image) using a procedure that adheres to thefollowing rules:

A pixel that is part of the Writings in none of the input images willnot be “written” in the composite image (i.e. it is assigned thebackground color, typically white).

A pixel that is part of the Writings in one or more input images will beassigned a non-background color. The choice of color depends on thecolor(s) of that pixel in the input images and is determined by a“merging algorithm.”

The “merging algorithm” used by the present invention takes a “layered”approach whereby the input images are is seen as (arbitrarily) ordered,overlapping layers. The output color chosen is the color of the pixel inthe topmost “layer” (input image) in which the pixel was “written.” Analternative approach would be to compute a new color for the outputpixel by blending the colors of the overlapping input pixels. Variousformulae may be employed for such blending and are known from theliterature. In any case, the “merging algorithm” typically applies to arelatively small portion of the pixels since the participants will nottypically overlap much of their Writings.

The current invention may be used for various applications other thaninteractive remote sessions. For example, the outgoing (or incoming)stream(s) of Local Updates could be recorded into storage [26] (possiblyaccompanied by other streams of data such as “raw” video and audio) forlater playback of a session. In this case a single site may operateindependently as a transmitter (during recording) or receiver (duringplayback) without taking Remote Update streams [305]. This independenceof transmit and receive modes is emphasized by the dashed lines in thelower part of FIG. 3.

Capture Engine [131]

The Capture Engine module [31] manages the flow of data and processingcontrol between the various modules of a transmitter system. As shown inFIG. 3, it is a high-level component that controls the operation of theReal-time Detector [32] and Capture Processor [33] sub-modules.

The Capture Engine [31] executes a repeating cycle of operations. Ineach cycle, it grabs a current Viewed Image [301] and also asynchronized and geometrically adjusted Computer Display Image (ifprojection is employed) [302] as described below. The input images arepassed to the Real-time Detector module [32], which determines whatrelevant changes, if any have occurred since the last cycle.Subsequently, the Capture Engine [31] may decide to invoke the CaptureProcessor module [33]. The criteria for this decision include: whetherthe Real-time Detector has found relevant changes, the time that haselapsed since the prior Local Update (as compared to a predefinedinterval giving the desired update-rate), the size of the detectedchange and the current processing load of the computer. If the CaptureProcessor [33] module is invoked, it produces an update that is passedto the Capture Codec module [34] for encoding and transmission.

After completing the cycle described above, the Capture Engine [31]schedules the execution of the next cycle. It determines an intervalduring which processing is suspended before beginning the followingcycle. The criteria for determining this interval include: whether ornot an update was produced in this cycle, the rate of detection mostsuitable for tracking changes and the processing load of the computer.Intervals will vary among implementations, but they typically range fromtens to hundreds of milliseconds in-between successive processingcycles.

As described above, the Computer Display Image used as input [302] (ifprojection is employed) should be “synchronized” with the Viewed Imageinput. This means that both images together simultaneously represent theappearance of the Writing Surface at a specific time. A counter examplewould occur if the Computer Display Image contained changes that did notyet appear in the Viewed Image or vice versa. Synchronization isrequired to avoid erroneous results. For example, if a newly projectedobject appears in the Viewed Image but the system uses an outdatedComputer Display Image that does not contain this object, the systemcould erroneously determine that this object is a new Local Marking.

Synchronizing the input streams to the desired degree of accuracy can beaccomplished using several approaches. In the preferred embodiment ofthe invention events that cause changes to the computer display aretrapped, and a queue of updates is maintained along with timinginformation. This enables the system to sample the Computer DisplayImage, as it appeared when the Viewed Image was sampled. If strictsynchronization cannot be implemented, the Capture Engine [31] can avoidprocessing when there is a risk of unsynchronized occurrences, forexample by skipping cycles when the operating-system or the CaptureCodec (in receiver mode) [34] indicate that the computer display isundergoing change.

Computer display information must also undergo Warping and“scaling”—operations that transform the geometry of the Computer DisplayImage to match that of the Viewed Image so that they can be comparedwith each other. Such transformation are well known in the art and canbe accomplished with high accuracy, for example by utilizing bilinearinterpolation, given the aforementioned calibration information andsystem parameters.

Real-time Detector [32]

The Real-time Detector [32] is responsible for tracking the informationcontained in the input streams (Viewed Images and optionally, theComputer Display Images) to determine, in real-time, if and whererelevant changes have occurred to the Writing Surface [21]. When suchchange, (Writings and Erasures) do occur, it updates internal image andstate information to reflect the detected change. Upon each execution ofthe Real-time Detector [32], it updates this state information whileperforming temporal integration to improve quality and reduce noise. Itoutputs a subset of this information that describes the currentlydetected change. This consists of one or more image masks indicatingwhich areas have changed and data from the Viewed Image thatcorresponding to those area.

The Real-time Detector [32] may be implemented as a series of three maindetection units:

1. Change Detector

This unit detects changes in the Viewed Image that are potentially newMarkings, but may also be caused by changes to the Projections (if any)or Viewed Image Interference. The unit distinguishes between LocalWritings and Local Erasures in order to use different processingtechniques optimized for each type of change. Image noise is alsoeliminated, for example by applying criteria of signal strength and sizeor shape constraints. In addition, results of prior cycles are used inorder to track the process of writing or erasing as it takes place.

The change detector is implemented using a combination of variousapproaches that are based on well-known image processing techniques.Processing phases include: comparing the Viewed Image input to thecurrent state information, including a Stored Viewed Image, to produce adifference image which is then binarized to determine the areas in whichsignificant changes have occurred, using edge-detection filters in orderto detect lines that represent new Writings or Erasures in those areas,gray-scale comparison operations to the known reference image or StoredViewed Images to verify and isolate the change, filtering of noise andother artifacts that are not actual lines and “stability checks” thatremove transient image artifacts by verifying that each change appearsin two or more consecutive cycles.

2. Interference Detector

This unit identifies image areas that contain Viewed Image Interferencesuch as a writer's body or clothing or other ephemeral objectsintroduced into the field-of-view. These areas are masked-off from theareas in which changes were detected in this cycle. An interferencedetector also employs a combination of standard image-processingtechniques. It compares the Viewed Image to the calibration referenceimage. The difference image obtained from this comparison is binarizedafter histogram analysis to determine an optimal binarization threshold.Morphological operations are used to smooth the result and close “holes”that appear in it. Low frequency components are assumed to be ViewedImage Interference. Transient changes are also assumed to be ViewedImage Interference and these are detected by “stability checks” thatcompare the current Viewed Image to the Viewed Image of the previousprocessing cycle. Additional clues to the existence of Viewed ImageInterference are obtained in areas in which Writings seem to disappear(yet an Erasure is not detected) and in areas in which Projectionschange appearance in the Viewed Image without a corresponding change tothe Computer Display Image. Additional accuracy may be obtained byemploying feature extraction and pattern recognition techniques toidentify certain, common shapes of expected obstructions such as auser's torso or hand. Processing color information can be useful indistinguishing skin or clothing. In addition, conclusions from priorcycles may be used in order to track motion of interfering objects.

3. Projection Cancellation (when Projection is Employed)

The purpose of this unit is to eliminate extraneous information from theViewed Image when such information results from projection of theComputer Display Image. Such Projections may include both a “fixed”(seldom-changing) Background Image as well as Remote Updates that arecontinuously being received from remote stations, if any such stationsare participating. The object is to leave only Writings and Erasuresmade on the physically local Writing Surface in the update undergoingprocessing. The motivation for projection cancellation is thus twofold:

It prevents the unnecessary transmission of projected backgroundinformation that is part of the “page” being displayed by the computer.This information is already available in digital form within thecomputer and should not be handled as if it were Markings on the WritingSurface.

It prevents transmission “echoes.” This means that each stationtransmits only information that was created locally without bouncingback projected Remote Updates received from remote stations.

The implementation is based on analysis of updates to the ComputerDisplay Image [302] and matching changes to it with changes to theViewed Image [301]. Wherever a matching change is found in both, it is“subtracted” or discarded. This leaves only the changes that representphysical updates to the Writing Surface [21] or an empty set when noactual changes were made. The subtraction is done in a careful mannerthat avoids, to the extent possible, the discarding of Markings thathappen to overlap Projections (for example, when a writer draws a linethrough a projected word). When overlap does occur, it is usually thecase that Markings are made over existing Projections or vice versa(rather than simultaneous appearance of both types of information). Bycomparing the Viewed Image to the Stored Viewed Image the new Markingscan be distinguished from preexisting Projections and the areas ofoverlap can be processed to appear similar and continuous to theneighboring non-overlapping areas.

The order of execution and precise division of responsibility betweendetection units may vary in alternate embodiments of the invention. Infact, they may even be programmed using multiple, interdependentprocessing stages since the conclusions of each unit may assist theothers in producing accurate results. For example, the “interferencedetector” may take advantage of clues produced by the “change detector”and “projection cancellation” units. Specifically, areas detected aspossible Erasures and areas in which the appearance of Projections isdistorted relative to their expected appearance may indicate thepresence of Viewed Image Interference. Conversely, the “change detector”and “projection cancellation” units could perform more accurately ifthey were supplied information about the location of Viewed ImageInterference in the Viewed Image as provided by the “interferencedetector.”

Capture Processor [33]

The Capture Processor [33] is responsible for transforming the ImageSensor data of changed Markings into aesthetically pleasing WritingImage data suitable for merging with other Writing Images and/or theBackground Image. Specifically, background areas that have no Writingsshould be colored white or some other predetermined or “transparent”background color. When applicable, Warping of the processed Markingsshould be performed to compensate for geometric distortions (see below).Markings should be readable, well formed and should reproduce the sameapproximate color as that of the marker used. Improving readability andform is accomplished by performing visualization enhancements, whichinclude “stretching” the gray-scale intensities and smoothing theMarkings using anti-aliasing techniques. Reproducing the marker color isachieved by performing “color normalization” and “color quantification.”The former utilizes information from the calibration reference image aswell as the Computer Display Image [302] (if projected) to overcomecolor distortions due to background lighting. Color quantifying assignsdiscrete hues to each given Marking, thereby reducing the number ofcolors in use. This serves to make the resulting updates more accurate,readable and compact. Additional operations may be performed such asvectorization and potentially, optical character recognition or othershape or pattern recognition operations. These may be intended tofurther improve the quality of the Markings for rendering, printing orother uses as well as to reduce the amount of data required so as toimprove compression. These steps utilize techniques that are based onextensive literature in the fields of graphics, image processing andpattern-recognition.

When Warping is possible (calibration information supplies it), theCapture Processor [33] Warps Markings in order to compensate forgeometric distortions of the Viewed Image due to the perspective oroptics of the Image Sensor device. This produces a straight, undistortedimage in a predetermined coordinate space which, preferably, is commonto all participating sites. An important result of Warping is that itbrings images from each remote site to the same geometry (or possiblydiffering only by a scaling factor if a common coordinate space is notenforced). This allows the images to be matched and merged as done insubsequent processing. Markings processed in this way, if projected,would accurately overlay the physical Writings that produced them.

Capture Codec [34]

The Capture Codec module [34] encodes local changes to Markings intocompact packets (Local Updates) that may be appended to a data streamundergoing transmission to other sites and/or to storage media [304]. Italso decodes such packets if they arrive from remote sites (RemoteUpdates) or if they are played back from storage [305]. Internalrepresentations of the local Writing Surface [21], if any, and of remoteWriting Surface [21] if any, are maintained for the encoding anddecoding operations. These are referred to as Writing Images. A merged,composite image of any subset of the participating Writing Images may beobtained from this module [306] as described earlier. Variousconsiderations affect the implementation of the encoding and decodingalgorithms used by the Capture Codec [34] One consideration is the dataformat(s) used to encode the image information. The choice depends,among other things, on the characteristics of the data, degree ofcompression obtainable and the amount of processing time required. Thecurrent invention utilizes the Microsoft RLE8 (8-bit Run LengthEncoding) format for encoding. This provides the advantages of a“standard” format and a reasonable degree of compression. Special codescould be employed to further improve the degree of compression. Giventhat the input data represents a stream of updates, each update may beencoded as a “delta” which specifies what information has changed in therespective cycle rather than an encoding the full representation of theWriting Image every time. As stated above, other types of packets areconceivable and are employed when advantageous and appropriate. Certainapplications require that a full-image encoding be periodically outputas a “key frame”, for example to allow a viewer to join in mid-sessionor for skipping to different positions when playing back a recordedsession.

Another consideration involves constraints on the output bandwidth (sizeof data produced per unit time). When bandwidth limitations are imposed,any Local Update [304] may be split such that part of it is output (upto the maximum allowed bandwidth) and part is buffered to be outputlater. To further optimize use of the allowed bandwidth, buffered LocalUpdates [304] are modified by subsequently produced Local Updates [304].This implies that Local Updates [304] for an area of the Writing Surface[21] that undergoes rapid successive changes may cancel each other outand reduce the overall output size (as a trade-off for the delay inbuffering). For example, when an area that has just been erased isimmediately overwritten (or vice versa), rather than producing twoseparate and conflicting Local Updates [304], only the net result willbe output, thus reducing the amount of bandwidth used.

Additional sophistication may be introduced into the Capture Codec [34]processing. Sub-units of the content of each packet may be prioritizedfor transmission based on various criteria so that “important”information arrives first, while less important updates are postponed aslong as necessary. Using another approach, an entire Local Update [304]can be transmitted, at first with poor quality but high compression(e.g., by sub-sampling to a lower resolution) and then graduallysupplementing the transmission with the missing information thatimproves quality. The motivation for such “progressive encoding” is toensure that the Local Update is transmitted, albeit with loss ofquality, even if limited bandwidth is available. In this case, new LocalUpdates [304] may take priority over enhancing the quality of priorLocal Updates [304], the latter being postponed until no new LocalUpdates [304] occur and bandwidth becomes available.

Detailed Examples

A detailed example of an image processing and display scenario ispresented in FIGS. 4A, 4B and 5. The diagrams of FIGS. 4A and 4B showsuccessive “snapshots” in time of the appearance of the Writing Surfaces[21] at three remote sites: transmit-and-receive sites A, B and C, andreceive-only sites D and E. Sites A, B and C have their composite imagesprojected upon their respective Writing Surfaces [21A, 21B and 21C].These snapshot representations of the appearance of these WritingSurfaces [21] follow a time line which progresses diagrammatically invertical columns down the page.

For purposes of illustration, assume site [21A] A's Writing Surface is alarge such as a standard, erasable whiteboard; at site B, a smallsurface such as a piece of paper; and at site C, a conventionalflipchart. As depicted in FIG. 1, the displays at sites D and E are aprojector [24D] and computer monitor [12], respectively, passivelydisplaying the composite images of FIG. 4B.

In the first illustration [41A], the writer at site A has drawn apicture of a fish on the local Writing Surface, upon which is projectedthe title “Fish Anatomy” (from the Background Image) throughout thesession. After a brief interval, the drawing of the fish appears at theother sites. [41C, 41D, 41E] (the appearance of the Writing Surface[21B] at site B is identical, but not represented). The writer at site Bthen annotates the projection of the fish drawing [42B] and theseWritings are soon visible at the other sites [42C, 42D, 42E] (theappearance of the Writing Surface [21A] at site A is identical, but notrepresented).

At this point, the writer at site A overwrites these projected Markingsto correct the error made by the other writer at site B [43A]. Again,after a brief interval, the written correction made at site A appears inthe Computer Display Images at Sites B,C, D and E [43B, 43D] (theappearance of the Writing Surface [21C ] at site C and the display [12]at site E are identical, but not represented). Subsequently, the writerat site C adds Writings as shown in [44C] and these appear at all othersites [44E] (all other surfaces and displays appear identical, but arenot represented). Thereafter, the writer at site B erases the incorrectMarkings (Local Erasure of the arrow) and replaces these with new,correct Markings (Local Writings of a different arrow) [45B]. Thesechanges at site B produce Local Updates, which appear at the other sites[45C, 45D, 45E] (the appearance of the Writing Surface [21A] at site Ais not represented). The process by which this particular Local Update[304] is produced is graphically depicted in FIG. 5 as explained infurther detail later.

Throughout the ongoing session, transient Viewed Image Interference(physical obstructions such as a hand or body) in the way of the imagesbeing captured are detected as non-informational and discarded prior totransmission to the other sites and/or storage. The result is acontinuous display at all five sites devoid of such distractions intheir respective Computer Display Images.

Finally, the writer at site A erases the prior Markings (site A LocalErasure) and makes new Markings in their place (site A Local Writings)[46A]. Soon thereafter at all five sites A, B, C, D and E, a replicaappears of the same composite image of the final result of this sessionof projections, Writings and Erasures at sites A, B and C. At site A,the Writing Surface [21A] contains the writer's physical Markingsinterposed upon the projected Computer Display Image [302] containingRemote Updates [305] from sites B and C. Conversely, at site B, theWriting Surface [21B] contains the site B writer's physical Markings,also interposed upon a projected Computer Display Image, [302] but thisComputer Display Image displays the Remote Updates [305] deriving fromsites A and C. At site C, the Writing Surface [21C] contains the site Cwriter's physical Markings, interposed upon a projected Computer DisplayImage [302], which, in this case, displays the Remote Updates [305]deriving from sites A and B. As receive-only sites, D and E did notparticipate in the alteration of the image; the Computer Display Image[302] at these sites is a composite of sites A, B and C.

Consequently, all five sites display what, in effect, arerepresentations of the same result, although the images actually differin two important respects: (a) the transmit-and-receive sites' viewerssee a combination of physical Markings and projections over them,whereas the receive-only sites view a display alone (i.e., no physicalMarkings); and (b) the scale and appearance of the images appearing maydiffer between the sites due to differences in the size of theWriting/Projection Surfaces [21] and/or resolution capabilities of theirrespective computer-driven display devices.

While the specific elements in each site's Computer Display Image [302]will vary according to where the Local Markings appear, and theresolution and size of the display may vary, viewers will neverthelesssubjectively perceive that they are seeing the “same” image, regardlessof which site they are at. Objectively, when at any time users at anysite elect to create a Committed Image [306], the process of the presentinvention will create a Committed Image [306] that will be identical atevery site, except for adjustments to accommodate differences inresolution and size of display.

By way of further explanation of the transformations depicted above,FIG. 5 illustrates the process in additional detail. Referring now toFIG. 5, there are depicted the processing steps and images used during atypical cycle of the software components shown in FIG. 3 that run on thecomputer of site B ([23B] in FIGS. 1 and 2A). Remote Updates [305A] and[305C] arrive from sites A and C, respectively, via the communicationsnetwork [25B]. Remote Updates [305A] and [305C] are used by the CaptureCodec [34] to construct internal representations of the physicalMarkings at each of these sites shown as Remote Writing Images [511] and[512], respectively. These remote images are merged within the CaptureCodec [34] to form a Composite Remote Writing Image [521]. If a computergenerated Background Image [307] is also employed, it too is merged withthe former to create a composite Computer Display Image [302], which isprojected onto the Writing Surface [21B] by the projector [24B]. At thispoint the visual scene contains, in addition to these Projections,Viewed Image Interference and the recently modified physical Markings onthe local Writing Surface [21B] as shown in [532] and [533]respectively. The Image Sensor device [22B] views the scene and providesa Viewed Image [301] containing a digital representation of this sceneincluding all the aforementioned visual elements—Projections, ViewedImage Interference and Markings. Due to the nature of the Image Sensorand its positioning, the Viewed Image contains both geometric distortionand limited optical quality and is degraded by noise and other knownimaging artifacts. The Viewed Image [301] is input into the CaptureEngine [31] along with the calibration Reference Image [303] and theComputer Display Image [302]. Assuming that the Viewed Image [301] andComputer Display Image [302] are properly synchronized (i.e. the formeris up-to-date with respect to the latter) the Real-time Detector [32]analyzes the images along with the Stored Viewed Image [542] and otherstored information not depicted in the figure, to determine whichcontent, if any, should be processed as changes to Local Markings. Itdetects and ignores both the Viewed Image Interference and Projectionsapparent in the Viewed Image leaving only changes to the Markings on theWriting Surface [21]. The Capture Processor [33] proceeds to clean-up,geometrically Warp and properly color the changed pixels, therebyupdating an internal Writing Image [551] with its results. Finally, theCapture Codec [34] compares the newly updated Writing Image [551] withthe previously Stored Writing Image [552] to determine precisely whichpixels have been written and/or erased since the last cycle ofprocessing. The result of this comparison is then encoded fortransmission via the communications network [25B] and/or for saving tostorage [26B] as a new outgoing Local Update [304], which is depicted inthe figure as a full image for the sake of clarity.

While the preferred and alternate embodiments of the invention have beendepicted herein in detail, modifications and adaptations may be madethereto, without departing from the spirit and scope of the invention asdelineated in the following claims.

What is claimed is:
 1. An apparatus for providing a portion of acomposite image on a first surface to a second surface remote from saidfirst surface, said apparatus including: (a) an image sensor forproviding a first signal indicative of images on said first surface; (b)a first computer for receiving said first signal and responsive to afirst stored signal to provide a differential signal; (c) a secondcomputer for receiving said differential signal and responsive to asecond stored signal to provide a second signal; and (d) a device forreceiving said second signal to provide said composite image to saidsecond surface.
 2. The apparatus of claim 1, wherein said first computerincludes an executable program in which said first signal is compared tosaid stored signal to provide said differential signal which is filteredto exclude signals indicative of interfering objects present in saidfirst signal.
 3. The apparatus of claim 2 in which said executableprogram further processes said first signal, said first signal having asuccession of frames, examining said succession of frames for variationsin said differential signal, thereby identifying said interferingobjects present in said first signal.
 4. The apparatus of claim 1,wherein said composite image comprises background information andmarkings and wherein the differential signal is filtered to exclude abackground signal from the composite image.
 5. The apparatus of claim 4,wherein said background signal is a projected signal, said apparatusfurther comprising a second device for providing said projected signalonto said first surface.
 6. The apparatus of claim 5, wherein theprojected signal changes periodically.
 7. The apparatus of claim 5,wherein the projected signal changes frequently and is indicative ofmarkings from additional remote surfaces.
 8. The apparatus of claim 1,further comprising a storage medium for storing said composite image,said composite image comprising images from said first and secondsurfaces at a given time.
 9. The apparatus of claim 1, furthercomprising a transmission interface for transmitting said compositeimage, said composite image comprising images from said first and secondsurfaces at a given time.
 10. The apparatus of claim 7, furthercomprising a storage medium for storing said composite image, saidcomposite image comprising images from said first, second and additionalremote surfaces at a given time.
 11. The apparatus of claim 7, furthercomprising a transmission interface for transmitting said compositeimage, said composite image comprising images from said first, secondand additional remote surfaces at a given time.
 12. An apparatus as inclaim 1, wherein said second surface is a display screen.
 13. Anapparatus as in claim 1, wherein said first and second stored signalsare the same signal.
 14. A method of providing a signal indicative ofmarkings made on a surface in a viewing field, which signal is notindicative of the presence of interfering objects should they be presentin said viewing field, comprising the steps: a. providing a signalindicative of said viewing field, said signal being composed ofsequential frames of said viewing field; b. detecting changes betweensuccessive ones of said frames of said viewing field to provide a seriesof changed viewing field signals; c. filtering said series of saidchanged viewing field signals to exclude signals indicative ofinterfering objects to provide a series of filtered, changed viewingfield signals; d. detecting changes between successive ones of saidseries of filtered, changed viewing field signals to provide said signalindicative of markings made on said surface.
 15. The method of claim 14,further providing a storage medium for storing at least one of saidseries of said filtered, changed viewing field signals.
 16. The methodof claim 14, further providing a transmission interface for transmittingat least one of said series of said filtered, changed viewing fieldsignals.
 17. The method of claim 14, wherein said viewing fieldcomprises background information and wherein said method furthercomprises the step of filtering said signal indicative of said viewingfield to exclude said background information.
 18. An apparatus forproviding a marked image onto a second surface, said marked imagecorresponding to markings on a first surface, said first surface havinga complete image thereon comprising background information and saidmarkings, said apparatus, comprising: an image sensing circuit forcapturing said complete image to provide a complete image signal; adevice responsive to a differential signal for providing said markedimage onto said second surface; and a unit for receiving said completeimage signal and responsive to a background signal for providing saiddifferential signal.
 19. An apparatus as in claim 18 wherein said secondsurface is a display screen.
 20. An apparatus as in claim 18 whereinsaid first and second stored signals are the same signal.